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Abstract 

In this note, we want to show (a) how the statistics of synaptic input 
influences the read-out of stored intrinsic properties for two-dimensional 
model neurons and (b) the effects of network topology on synchronization, 
i.e. the correlations of synaptic input generated in the network. This the- 
oretical work relates to the biological properties of neuromodulation by 
presynaptic and postsynaptic effects on connectivity and intrinsic prop- 
erties (0). We have previously found that highly synchronous inputs 
drives neurons, such that small differences in intrinsic properties disap- 
pear, while distributed inputs lets intrinsic properties determine output 
behavior (0). We choose a parametrizable two-dimensional neural oscil- 
lator model (similar to a Morris-Lecar model), to investigate the effects 
of changed network topology for larger collections of neurons. We focus 
on intrinsic properties that encode frequency-specificity. 

We suggest that presynaptic modulation can be an effective way of 
rapidly altering network topology. We investigate changes in network 
topology along the dimensions of small-world connectivity vs. random 
graph connectivity. We hypothesize that SW graphs produce more glob- 
ally synchronized feedforward behavior with lower link density than com- 
parable random graphs. In accordance with the hypothesis, we find that in 
a SW graph, because of highly synchronous inputs, the difference between 
neuronal intrinsic properties is minimized, while a random graph allows 
read out of neuronal intrinsic properties. Thus, altering network topol- 
ogy can alter the balance between intrinsically vs. synaptically driven 
determined network activity. 



•Based on a presentation at the AMS Meeting on Dynamical Neuroscience, March 2005, 
Santa Barbara. Thanks to E. Izhikevich and R. Hecht-Nielsen for helpful discussions. 
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1 Introduction 



We have previously shown for a conductance-based neural model of striatal 
medium spiny neurons that neuronal variability in the contribution of indi- 
vidual ion channels (such as slowly inactivating potassium channels and GIRK 
channels) can yield uniform responses, if the neurons are driven with sufficiently 
strong synaptic input fluctuations (see If the same neurons are driven by 
more distributed synaptic input, they show variability in their response pat- 
tern, such as spike timing and spike rates. These different statistical properties 
of synaptic input can be modeled by a variability in the correlation properties of 
input neurons. I.e. driving neurons by correlated vs. uncorrelated input leads to 
uniform behavior vs. read-out of stored differences in ion channel conductances, 
(see 0). 

We shall explore the effect of different network topologies on the statistics 
of synaptic input received at individual model neurons. The hypothesis is that 
certain topologies create more synchronized input volleys than others. 

2 Methods 

2.1 Ion channel based neuron model and synaptic input 

The conductance-based neural model of striatal medium spiny neurons is de- 
scribed in detail in W . The membrane voltage Vm is modeled using the equation 

Vni = + ^2(^2) • • • + Ain(^) - Isyn] 

where the h are the currents, induced by the individual ion channels. Variabil- 
ity of the neuron is modeled by modifications to /x;. This model includes ion 
channels for Na {In a), K (Ik), slow A- type K channels {I As), fast A- type K 
channels {Ia/), inward rectifying K channels {I Kir), L-type calcium channels 
{Icah), and the leak current (Iieak)- The definition of all parameters and the 
dynamics of the ion channels can be found in |5j . 

For the experiments in this paper, we only focus on variability, induced 
by changes in the strength of the slow A-type K channels. The total current 
contribution for these channels is (iIas where fj, was selected between 1.0 and 
1.5. 

In order to illustrate the variability in neuron behavior, we excited the neuron 
model by input signals, resembling two kinds of synaptic input: uncorrelated 
and correlated. These signals were generated by superposition of excitatory 
and inhibitory spikes generated by individual Poisson-distributed spike trains 
(50 excitatory and 10 inhibitory), and biased Gaussian background noise. 

The amount of pairwisc correlation in these spike trains governed the type 
of input signal. A high correlation factor was used to generate sequences with 
distinct short periods (10-15ms) of high activity. 
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2.2 Variability in a two-dimensional model 

In order to do large-scale simulation we need to employ a simple, computa- 
tionally tractable neuron model. We use a two-dimensional model of a neural 
oscillator (cf. Morris-Lecar model, Fitzhugh-Nagumo model, We employ 
an instantiation of the model with parameters fitted to the general properties of 
cortical pyramidal neurons (see This model consists of an equation for the 
membrane model v (Eq. ^ , fitted to experimental values for cortical pyramidal 
neurons (Eq. and an equation for a gating parameter u (Eq. |2J|. 

i! = OMv"^ +5V+U0-U- Isyn (1) 



u — a{bv — u) (2) 
b = 0.2; a = 0.02 

When the neuron fires a spike (defined as v{t) = 30 mV), v is set back to a 
low membrane potential v := c; c = —65.8 mV and the gating variable u is in- 
creased by a fixed amount d {u :— u + d, d = 8) (cf. jSj). This formulation allows 
a very simple neuron model, which avoids the explicit modeling of the downs- 
lope of the action potential, and rather resets the voltage. Time-dependence is 
modeled by the gating variable u. 

Neuronal variability is achieved by systematic variation of inactivation pa- 
rameters. By varying d, we can vary the inactivation dynamics of the model 
after a spike, by varying a we vary the inactivation dynamics throughout the 
computation. In this way, we can attempt to model neuronal variability in acti- 
vation/inactivation dynamics, which is sufficient to model frequency-selectivity 
as an example of a stored intrinsic property (see Table ^ . 



par am 


a 


b 


c 


d 


generic 


0.02 


0.2 


-65 


8 


typel 


0.025 


0.2 


-65 


6 


type2 


0.02 


0.2 


-65 


9 


type3 


0.015 


0.2 


-65 


12 


type4 


0.015 


0.15 


-65 


14 


types 


0.022 


0.3 


-65 


14 


type6 


0.022 


0.3 


-65 


9.5 



Table 1: Parameters for different neuron types 



2.3 Graph properties 

We create graphs of N excitatory and 10% inhibitory neurons. We change the 
connectivity pattern for the excitatory neurons and keep the inhibitory neurons 
fully connected to all excitatory neurons. We compare randomly connected 
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(RG) to small- world graphs (SWG). A randomly connected graph is fully spec- 
ified by N and K, the number of links. 

A SW graph can be created using linear preferential attachment (LPA, U): 
the graph is generated one vertex at a time, and each newly created node is 
attached to a selection of d already existing nodes. The probability of selecting 
a specific node is proportional to its current degree. For our experiments, we 
use an efficient algorithm found in |7|. 

There is a useful overview of this and other methods in 0] . 

We use specific instantiations of these graphs (RGl and SWGl) for the 
simulations. 

The SW graph was created by LPA which resulted in K — 1822 connections 
{N = 210), the random graph had the same number of neurons {N = 210) and 
approximately the same number of connections K = 1800. Table|2shows global 
graph characteristics for random graphs (RGl, RG2) and small-world graphs 
(SWGl, SWG2). A SW graph is defined by (a) high clustering (b) a power-law 
distribution of degrees and (c) a short average path length. It is noticeable 
that the differences in mean path length are small, probably because of the 
small size of the graph, while the cluster index is three times higher for the SW 
graph. These are standard measures for "small- world like". Different networks 
can be sorted according to these and possibly other numerically defined prop- 
erties. Fig. [TCI ITSl show the distribution of outdegrees (for degrees < 25). The 
distribution is that of a power-law vs. a Gaussian distribution, in accordance 
with the definitions. Fig. 1131 shows a 40-neuron subgraph from each structure. 
The higher clustering index in the SW graph is apparent. 



Property 


random graph(RGl) 


small-world (SWGl) 


RG2 


SWG2 


N 
K 

indegree 
outdegree 
cluster index 
mean plength 


210 
1800 

8.57 [2 ... 20] 
8.57 [1 ...19] 
0.041 
2.72 


210 
1822 

8.67 [1 ...25] 
8.67 [0 . . . 142] 
0.12 
2.46 


210 
1800 

8.57 [2 ... 19] 
8.57[2 ...19] 
0.043 
2.71 


210 
1815 

8.64 [0 . . . 13] 
8.64 [0 ...131] 
0.1 
2.35 



Table 2: SW vs. Random Graph Properties 



2.4 Define Correlation Measure 

We also define synchronization S" in a graph by the total of pairwise correlations: 
for each pair of neurons ni, n2, we count the number of spikes which occur within 
a window W from each other (here W = 10ms) and divide by the total number 
of spikes for ni or n2 respectively. We then add up all non-diagonal values in 
the matrix, and divide by the number of non-diagonal matrix elements to arrive 
at an average pairwise correlation value 5*. This correlation measure does not 
take higher-order correlations into account - which does not seem significant for 
the comparison of the results in this case. 
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3 Results 



3.1 Frequency-specificity as a stored intrinsic property 

We selected frequency-specificity as a model case for a stored intrinsic property, 
defined as regular spiking behavior with a fixed frequency, i.e. an average ISI 
with a low standard deviation. 

First, we illustrate this with a full ion channel based model (the MSN model, 
and variation in the slow A- type potassium channel {I As)- We also use two 
types of synaptic input, correlated and uncorrelated, in accordance with the 
definitions in j^. 

In Fig. 121 (left), we show the response of MSN model neurons with a scaling 
of HiAs — 1-0, 1.3, 1.5 to an uncorrelated input signal. 

The top panel shows the development of Vm over time. The middle panel 
shows the spike-train for each neuron with the mean ISI and its standard devia- 
tion; the total number of spikes is shown on the right. The bottom panel shows 
the input. The dots correspond to the spiking events for neuron 3 {hias — 1.5). 
The resulting mean ISIs are 25, 37, and 45 ms. With a of 6, 11, and 8, they 
are clearly distinguishable. This is also shown by the Gaussian distribution for 
the frequencies for each neuron type (Fig. O (right)). 

This model shows frequency-specificity as read-out of the density or relative 
contribution of the slow A-type potassium channel, indicated by the scaling 
factor ^lAs ■ To emphasize the conditional aspect of this behavior, we compare 
this result by stimulating the same neurons with highly correlated input (Fig.Q. 
Here we see a time-locked spike pattern which is expressed by similar mean ISIs, 
and an overlapping frequency distribution (Fig.^ (right)). 

3.2 Differentiated responses of neurons to synaptic input 
correlations 

In the generic two-dimensional model we find the same behavior: 

1. for correlated input, neurons spike time-locked to peaks in the input 

2. for uncorrelated input, a mean ISI length with low standard deviation 
(frequency-selectivity) emerge 

We searched for additional parameter values to define a number of different 
neuronal types with different mean ISI length. The best results were achieved 
with o and d varying in the opposite direction (see Table 

Clearly, a more extensive parameter search may also yield continuous vari- 
ability in frequency-selectivity. In particular, we may regard such a search as 
an exercise in discovering those regions in parameter space that make the most 
difference in input response. Since we are only working with a highly artificial 
neuron model, where parameters have no direct correspondence to reality, this 
is not further pursued in this paper. It would be interesting to rephrase this 
problem in the context of ion channel types and their effects on neurons. 
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We show the response of type 1-3 model neurons to more or less correlated 
input in Fig.|5land Fig.|H| 

We also show the response to driving type 4-6 neurons with uncorrelated and 
more irregularly timed correlated input (cf. Fig. El • When the driven input 
has different ISIs the standard deviation goes up, and the Gaussian distributions 
are broader. 

For the behavior of a large number of neurons, we show the difference of 
using uniformly defined, "generic" neurons, vs. variable neurons with slight 
differences in their parameters. In Fig. |51 (left), highly correlated input results 
in synchronous firing, while distributed input results in asynchronous, variable 
firing (see Fig.|5Jright)). When we use only three different types of neurons and 
order them in large stripes, the frequency-selectivity becomes visually apparent. 
The respective frequency distributions are shown in Fig. 1101 

3.3 Synchronicity is enhanced in a SW network 

In the following experiments, we explore the influence of graph structure on 
correlations within the graph. In Fig. ^] we see the activity in a SW graph 
(SWGl) after 1000 ms of initial stimulation to 10 neurons. There is also a 
small background inhibition to all neurons. Visual inspections suggests high 
synchronization in the graph. Fig. ?? shows the full simulation (5000 ms). The 
overall correlation, defined by pairwise correlation of neurons, is given in table 
131 Thus the Gaussians should look like in Fig. ^| (left), strongly overlapping. 
In Fig. ^2 (right) we see that this is indeed the case. 





RGl 


SWGl 


s 


0.22 


0.53 


S(Gen) 


0.36 


0.63 


S(Typel) 


0.28 


0.67 


S(Type2) 


0.12 


0.53 


S (Types) 


0.17 


0.62 


S(Type4) 


0.21 


0.60 


S (Types) 


0.25 


0.61 


S(Type6) 


0.20 


0.60 



Table 3: Synchronization measures for different neuron types and graph types 

In Fig. El we see the randomly connected graph (RGl) with the same initial 
stimulation. Background inhibition is the same as before. Visual inspections 
suggests asynchronous behavior. Pairwise correlation should be lower than be- 
fore, at least between different types. Again, Fig. 1151 shows the full simulation. 

The degree of frequency-specificity can be assessed by the average and stan- 
dard deviation for ISIs, as shown in the Gaussian. Except for the generic neuron 
g, we get separated frequencies, even though the neuronal types have not been 
optimized for this task. 
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3.4 Driver (Sources) and Transfer (Resonator) Modes 

The pairwise synchronicity S could be shown to be fairly independent of the 
properties and frequency-selectivity of the neuron. This is in accordance with 
the observation that differences in intrinsic properties disappear as the syn- 
chronicity in a network increases. Instead S is dominated by the type of net- 
work connectivity. If there is a phase transition in synchronicity during a grad- 
ual transformation of the graph structure, this would indicate a self-organized 
switch between time-locked behavior and read-out. 

Let us explain a certain case in the model: if neuron A receives less organized 
synaptic input, its intrinsic properties will determine its firing rate more. It will 
be more of a "driver" neuron, whose stored properties determine its output. If 
the same neuron A receives synaptic input that puts it more into a "driven" 
mode, its behavior will be more like that of a "transfer" (resonator) neuron, or 
in other words, a more uniform processing device that transmits information 
according to a simple transfer function 

4 Discussion 

4.1 Conditional access of stored properties 

Small differences in intrinsic properties are interesting for models of memoriza- 
tion, because they allow conditional read-out of information. This means that 
neurons may acquire differences in their ion channel distribution and density 
in a use-dependent way, storing aspects of previous activations. This memory, 
however, is not always present to influence all types of processing. Rather, it 
is conditionally present, i.e. it requires specific conditions to be "read-out". 
This observation provides a new perspective on the role of neuronal circuits in 
encapsulating memory (see 

Difference in neuronal read-out may not be restricted to synaptic input that 
is highly correlated vs. more distributed. Another type of synaptic input that 
has been shown to influence variability in neuronal spiking behavior is balance 
of inhibitory and excitatory input. This is another method of influencing the 
state of the neuron such that it either responds only to strong synaptic input, 
i.e. exists in a mode where its output is determined by synaptic input (in a 
low balance, low inhibitory state), or exists in a state where it is close to firing 
threshold (or "high-conductance state"). If we assume a model neuron with 
a certain type of nonlinear dynamics, then the synaptic input in this balanced 
state becomes a form of turbulence that causes a neuron to emit spikes according 
to its own stored intrinsic properties. 

4.2 "Symbolic" encoding by frequency specificity 

When we have frequency-encoding neurons with different rates and a tendency 
to fire regularly when stimulated by noisy, asynchronous network activity, we 
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can use these neurons for a form of " symbolic" encoding (i.e. assigning different 
symbols to the frequencies). 

We need to take the existing variability concerning the regularity (the stan- 
dard deviation) in our model into account, plus the observation that the de- 
viation is larger for sequences of 1000 ms or less, but fairly small for larger 
sequences such as 5000 ms or more. However, it should not be difficult to build 
reliable storage devices based on a distribution of deviations/frequencies such 
as for the given model neurons. 




Figure 1: Possible subgraph (tree) in a network. Different frequencies are indi- 
cated by the letters A and B. 

As an example, consider the subgraph displayed in Fig. ^ When activated 
at point O, assuming a traveling wave in three steps, and assuming different 
frequencies for A and B, we would get a schema as in Fig.|21 which would result 
in different actual timings, depending on a number of additional variables. 

We can also combine neurons in specific arrays (possibly including delays) 
and thus generate specific regularly repeatable spike-time patterns. As a matter 
of fact, using frequencies as symbols and then stimulating specific neurons such 
that sequences of neurons are being activated will generate spike-time patterns 
as mixtures of symbols 

It is entirely possible that this model does not capture reality very well. It 
would predict that certain neurons in a group fire with a specified frequency, 
when they receive distributed synaptic input, but join in a time-locked firing 
mode at other times. This would be difficult to establish empirically. The spike 
firing patterns that we can create with this basic generation model are so various 
that it should be possible to match them to existing recordings in any case (such 
as [?]). 
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Figure 2: Schematic view of expected spike timings using the subgraph, fre- 
quencies and numbering of the nodes from Fig. ^ The actual distribution of 
spike timings could vary considerably. 

4.3 Further questions 

There is no doubt that this basic model raises further interesting questions. 
When a pattern of activity is activated in a certain brain area, this may drive 
synchronized activity, time-locked to this pattern in other brain areas. Any 
actual read-out requires not only a localized activation pattern, but also mech- 
anisms for recruitment of other neurons and enhancement of the pattern by 
(synchronized) neural activity. Similarly, the question as to how the informa- 
tion is encoded from ongoing network activity into individual neurons is also 
not answered here. 

We may also consider the question of whether a neuron cannot simultae- 
nously respond to an input and read out its stored pattern. This would be 
the case for phase delay or advance by selective individual inputs. (The im- 
pact of such phase delays or advances is different if these are single events, in 
which case they only distort a specific processing event, or if they amount to 
phase resetting; or even to phase resetting with continuation at a different fre- 
quency.) There may also be other modes of combining input sensitivity with 
stored properties. 

The results we obtained were done with two-dimensional neurons, as models 
of cortical pyramidal neurons. We used both a generic model, and parametriza- 
tion according to a few distinct types. In this way, we can demonstrate a "sym- 
bolic" property of storage in distinct frequencies that a neuron emits, when 
stimulated in a noisy, distributed way. Nonetheless it could be interesting to 
perform simulations with more continuous variability over a range of parame- 
ters. We expect synchronization properties to be largely unaffected by this. The 
interesting question would be to interpret more continuous frequency ranges, if 
these exist. 
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4.4 Background inhibition 

The synchronization measures given in this graph are not further modulated 
by rhythmically structured, fast inhibitory spiking. We did not discuss feed- 
forward inhibition by fast-spiking interneurons, which are usually assumed to 
play an important role in narrow synchronization for high-frequency (> 20Hz) 
oscillations. This is an open question for further research. 

We have not systematically varied the amount of excitatory/inhibitory back- 
ground noise, nor specific types of background activity beyond white noise - the 
interactions with the results presented here could be suggestive. 

We have worked with short-term synaptic stimulation and additional back- 
ground noise to keep activity going. There is thus "re-entrant" self-supporting 
activity to the driven input. 

It is possible to put the network into a mode where the total activity is too 
low to support ongoing activity and the network needs ongoing outside input. In 
this case the self-organization properties reported here would undergo qualified 
alterations. For instance, input to hubs could have different results than input 
to other neurons. Changes of overall activity level in the network could also 
differ with changes in topology, and they would invoke homeostatic (control) 
corrections to keep activity constant. 

4.5 Synapses and graph structure 

We also need to clarify the relationship between synaptic strength of in-and 
outputs and degree of in- and outputs. The effective connectivity of a neuron can 
be defined by a combination of synaptic strength and degree of connectivity. We 
will thus arrive at capacitated graphs, i.e. the graph structure is "implemented" 
by synaptic weights onto link connectivity. Based on experimental estimates on 
the distribution of presynaptic receptors, we may also assume that each topology 
change incorporates between 10-20% of connections. 

The concept of SW graphs in this context needs to be further explored. 
Potentially important features which need to be quantified to be employed as 
measures are for instance the incidence of recurrent loops and the incidence of 
laterality under the assumption of a clustering of a SW graph into imperfect 
trees (hierarchical clusters). Differential stimulation to prime neurons (hubs) 
vs. subsidiary neurons (leaves) in a SWG also needs to be explored. This has 
impUcations for information storage (see 14.21 as well). 

5 Conclusion 

We have made some progress in this paper in clarifying the relationship between 
intrinsic frequency encoding, synchronization and a generic model of small- 
world networks. Applications to a real model of cortico-striatal interactions 
are planned. Also, some of the questions raised may merit further systematic 
treatment. 
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Figure 3: (Left) Frequency response of MSN model with variable scaling of Ias 
to uncorrclatcd input (Right) Gaussian distributions of ISIs. We see a clear 
separation of frequency responses 




Figure 4: (Left) Frequency response of the same MSN model to correlated input 
and (Right) Gaussian distribution of frequencies. We see less clear separation 
(overlapping) of frequency responses 
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Figure 5: Frequency response of two-dimensional model (type 1,2,3) to imcorre- 
lated input(left) and Gaussian distribution of frequencies 49 (typel), 63 (type2), 
90(type3) (right) : clear separation of frequency responses. 
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Figure 6: Frequency response of two-dimensional model (type 1,2,3) to corre- 
lated input (left) and Gaussian distributions 47 (typel), 48 (type2), 52(type3) 
(right) : overlapping of frequency responses 
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Figure 7: Response of neurons type 4,5,6 to uncorrelated input (left) and the 
corresponding frequency distributions (right) 
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Figure 8: Response of neurons type 4,5,6 to irregularly timed correlated input 
(left) and the corresponding frequency distributions (right) 
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Figure 9: 100 generic neurons (top), 100 neurons of type 1,2,3 (middle) respond 
to synaptic input (Left) Response to driving input (Right) Response to dis- 
tributed input (generated in the same way for each neuron but with different 
random values) 




100 120 140 



Figure 10: (Left) Gaussian curves for ISIs for correlated input (Right) Gaussian 
curves for uncorrelated input 
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Figure 11: Synchronization in a SW graph (SWGl) with variable neuron types; 
(left) rasterplot from 1000-2000ms (right) frequency distributions. Pairwise cor- 
relation in graph is S' = 0.53 



100 200 300 400 500 600 700 80O 800 100O " 40 60 80 100 120 140 160 180 200 220 

Figure 12: Asynchronous behavior in a random graph (RGl) with variable 
neuron types: (left) rasterplot from 1000-2000ms (right) frequency distributions. 
Pairwise correlation 5 = 0.22. 
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Figure 13: (Left) 50 neurons from SWGl and their connections (Right) 50 
neurons from RGl and their connections. The higher clustering in the SWG is 
apparent. 
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Figure 14: Full simulation for SWGl. 1 s stimulation, followed by 4 s ongoing 
activity 
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Figure 15: Full simulation for RGl. 1 s stimulation, followed by 4 s ongoing 
activity 




Figure 16: Outdegree histogram for RGl (red) and SWGl (blue) (fc < 25). 
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Figure 17: Full outdegree histogram for SWGl, SWG2, RGl and RG2. 




Figure 18: Outdegree histogram for SWGl, SWG2, RGl and RG2 (fc < 80). 
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